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Dear Reader, 


In its report titled Quality and Opportunity, Vision 2000 offered a wide range 
of Proposals for how Ontario's college system might meet the challenges of 
moving into the twenty-first century. In endorsing the recommendation of 
Vision 2000 for the development of a College Standards and Accreditation 
Council (CSAC) the Government of Ontario created the CSAC Establishment 
Board to provide the Minister of Colleges and Universities with advice on 
how CSAC should be implemented. 


Continuing the process of wide consultation exemplified in Vision 2000, this 
discussion paper presents, for your consideration and response, the Establishment 
Board’s proposals for CSAC. We look forward to reading your responses and to 
discussing this paper with many of you in the weeks ahead. We would appreciate 
written submissions not later than May 8, 1992. 


We believe that the proposed College Standards and Accreditation Council 
can be a focal point for fostering excellence in programming in Ontario 
colleges. If CSAC is to succeed, it will be because, collectively, we work for its 
SUCCESS. 


Let us begin by ensuring that the CSAC we create is a CSAC which will meet the 
needs of both internal and external constituencies to our college system. Toward 
this end, we welcome your responses to this paper. 


Sincerely, 
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Richard Johnston rnard Shapiro 
Chair Deputy Minister 
Council of Regents Colleges and Universities 
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Introduction: The CSAC Establishment Board and Guide to the Paper 


In May 1990, the Council of Regents forwarded the final report of Vision 2000 

- Quality and Opportunity - to the Minister of Colleges and Universities. The 
report on the Province's twenty-three community colleges contained forty 
recommendations covering a wide variety of issues. Among the most 
important of the initiatives were those related to enhancing the quality of 


pees offered by the college system. Four specific recommendations were 
offered: 


There should be a significant increase in the generic skills and general 
education content of programs leading to a college credential to ensure an 
equivalence of learning outcomes between these components and specific 
occupational skills. (Recommendation #2) 


There should be system-wide standards for all programs leading to a college 
credential. Such standards must focus on the learning outcomes expected of 
graduates from a program. (Recommendation #3) 


All programs leading to a college credential should be subject to regular, 
system-wide program review for the purposes of accreditation. 
(Recommendation #4) 


A College Standards and Accreditation Council (CSAC) should be established, 
with participation of internal and external stakeholders and with executive 
authority in the areas of system-wide program standards, review and 
accreditation. (Recommendation #5) 


Some of the goals underlying these recommendations were the following: 


(a) to enhance both the quality of and respect for a college credential and to 
assure a common meaning for credentials; 


(b) to increase the emphasis on generic skills in order that graduates of all 
college programs be able to exhibit the skills necessary not only to 
perform a particular job, but to engage in additional education and 
training as required and desired; 


(c) to increase the emphasis on general education in order to increase college 
graduates’ understanding of themselves and their society, to enhance 
their abilities to formulate and achieve personal goals and to enhance 
their abilities to function effectively as members of society; 
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(d) to ensure consistency through the application of system-wide standards 
and program review, applying to all similar programs regardless of 
where they are offered, how they are organized (full- or part-time) and 
how they are delivered; 


(e) to ensure that standards are current, appropriate to employers and 
learners, and to ensure that they are being met; 


(f) to establish processes for developing system-wide standards and 
conducting program review that do not lead to standardization; each 
college will determine specific curricula and modes of delivering 
programs and the mix of offerings available to learners; 


(g) to facilitate lifelong learning by allowing students to accumulate and 
transfer credits between part-time and full-time programs, between 
programs and/or institutions without significant loss of credit, thereby 
promoting accessibility; 


(h) to provide, through CSAC, the means by which the college community 
can assume ownership of quality on a system-wide basis and exercise its 
responsibilities in partnership with relevant stakeholders. 


Following initial examination of the recommendations, the Minister 
determined in January 1991 that the Council of Regents should convene an 
advisory committee to examine some of the issues which would arise in 
pursuing a College Standards and Accreditation Council. Following the 
meetings of this committee in the Winter and Spring of 1991, the Treasurer of 
Ontario announced, in the April 1991 Budget, the Government's 

commitment to proceed with CSAC. In September 1991, the Minister 
announced the creation of a CSAC Establishment Board to be co-chaired by 
the Deputy Minister of Colleges and Universities and the Chair of the Council 
of Regents, and to include representatives of the major college constituencies, 
Council of Regents, Ministry of Colleges and Universities and members 
external to the colleges.! The Minister's announcement and terms of 
reference? for the Establishment Board request that the Board submit its 
advice by June 1992. 


The Establishment Board held its first meeting on November 4, 1991, and 
concluded early in its deliberations that it would benefit from the input of the 
college community, broadly defined to include both internal and external 
stakeholders. This discussion paper is one vehicle toward this end. In 


1 A list of Establishment Board members is found in Appendix A. 
2 Attached to this discussion paper as Appendix B. 
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addition, the members of the Establishment Board have sought the views of 
their constituencies on the major issues facing CSAC. 


At the seven meetings to date, the Establishment Board has examined a 
variety of documents, including those prepared for the COR advisory 
committee and those prepared by the Board secretariat. The Board has 
discussed the issues of a mandate and functions for CSAC, general education 
and generic skills, program standards and program review. In the course of 
its deliberations, the Establishment Board attempted, wherever possible, to 
reach consensus in an effort to present a discussion of how CSAC might 
address the major issues of interest to both members of the Board and many 
within the college community. While the Board initially felt that the 
discussion paper might present alternative models of CSAC, its conclusions 
on many of the issues has led to the presentation of a single view of CSAC 
and how it might operate to fulfill its mandate. The discussion paper does, 
however, attempt to provide an account of the main lines of discussion and 
rationales for the Establishment Board's perspective. 


The discussion paper is divided into five sections. Section I proposes a 
mandate for CSAC in the form of five declarations, followed by some notes of 
clarification of the first declaration on CSAC’s jurisdiction. Section II 
elaborates on the other declarations in discussing the principal functions of 
CSAC: setting program standards and overseeing program review, and 
includes the Establishment Board's perspective on the central issues of 
general education and generic skills. Section II outlines the internal 
structure of CSAC as well as its relationships with its major constituencies 
and external bodies. Section IV provides some preliminary understanding of 
CSAC's priorities, the staging of implementation, pilot projects and some 
costs associated with CSAC. A final section summarizes the questions which 
the Establishment Board feels are most necessary for the college community 
to address. ) 


At the time of writing this discussion paper, two additional factors have been 
understood to have potential impact on the implementation of CSAC. The 
recommendation in Vision 2000 to provide a significant increase in general 
education and generic skills represents a major re-orientation of college 
programs with which the Establishment Board concurs. However, in 
discussing how to implement this and other commitments, the Board has 
attempted to consider the extent of financial constraint the government and 
the colleges must face. The Establishment Board considers its task to include 
the need to provide the Minister with advice about how CSAC can be 
implemented and meet its mandate in the most efficient way possible. 


A second, and related factor, is the government's commitment to proceed 


with the Ontario Training and Adjustment Board (OTAB). This body 
represents a joint initiative of labour and management to organize and 
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augment the educational and training activities offered by a variety of 
providers, under the auspices of both Federal and Provincial governments, to 
meet the many and diverse needs of the residents of Ontario. The colleges 
have historically played an important role in the delivery of a wide variety of 
adult education and training programs. For this role to continue to grow, 
especially in light of expenditures which may not grow, the college system 
must be able to demonstrate its ability to provide programs of the highest 
quality. The potential benefits of establishing CSAC may therefore be 
considerable for the well-being of the colleges and their ability to serve the 
needs of our labour force, both present and future. 


These are clearly difficult and challenging times, and not only for the colleges. 
Realizing the title of the Vision 2000 report -- Quality and Opportunity — will 
not be easy. The Establishment Board considers CSAC an important element, 
and we have tried to provide an outline of both what CSAC can be and how 
we might realistically proceed, cognizant of the difficulties to be faced. It is 
our hope, therefore, that readers will consider this paper carefully and | 
respond thoughtfully to the directions proposed. 


There are three principal means by which the Establishment Board is seeking 
responses to this discussion paper. First, a meeting with one of the 
Establishment Board co-chairs has been scheduled at each of the twenty-three 
colleges in April and early May at which time individuals from across the 

- college are invited to speak directly to the issues raised in this paper. Second, 
the Board will explicitly seek the responses of a range of external groups, 
including those which currently have formal responsibility for accrediting 
specific college programs. Third, individuals and groups, both internal and 
external to the colleges, are invited to respond in writing to the issues raised 
in the discussion paper. These responses should be received, not later than 


Friday, May 8, 1992, by: 


CSAC Establishment Board 
c/o Ontario Council of Regents 
10th Floor 
790 Bay Street 
Toronto, Ontario 
M5G 1IN8 
(fax) 416-325-1792 
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Section I: Mandate of CSAC 
I.A. Declarations 


1. CSAC SHALL have the authority to define (non-degree) credentials for, set | 
standards for and accredit publicly-funded college programs. 


2. CSAC SHALL have the authority to approve for each program a system-wide 
program standards document which will specify learning outcomes for 

- both generic skills and program-specific vocational skills, and any specific 
admission requirements. 7 


3. CSAC shall have the authority to define general education and to require 
that each college credential include a defined proportion of general 
education. 


4. CSAC SHALL oversee system-wide program review. Each program will be 
reviewed regularly. The purposes of program review will be to determine 
(1) the extent to which program outcomes which have been established 
have been achieved by students and (2) the appropriateness of program 
outcomes. Following the system-wide review of a program, CSAC will 
determine whether the program will continue to be accredited at a college 
and any conditions which might apply. The accreditation status of the 
program will be publicly reported. 


5. CSAC SHALL operate in an open and democratic manner, with an equal 


number of representatives from both external stakeholders and internal 
members of the college community. 


Are there other matters which should be included in CSAC's mandate? 
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I.B. Discussion of Jurisdiction 


The first declaration reflects the opinion of the Establishment Board that 
CSAC should have jurisdiction over all publicly-funded college programs. 
Publicly-funded programs offered in colleges include: 


¢ post-secondary (including one year certificate, two and three year 
diploma, and post-diploma programs); 


e adult training (including those purchased directly by the federal and 
provincial governments and indirectly by the federal government); 


e adult basic education including English as a Second Language, Ontario 
Basic Skills and FUTURES Pre-employment programs; 


¢ in-school portion of apprenticeship training; 


e Trades Updating and Technician and Technologist Updating. 


The Board considered alternative mandates for CSAC including 
e limiting CSAC's jurisdiction to post-secondary programs, or 


¢ expanding CSAC's jurisdiction to all programs regardless of source of 
funding. 


In support of restricting CSAC's mandate to post-secondary programs, the 
Establishment Board has noted that setting standards for and accrediting post- 
secondary programs is a large and complex task in itself. The quality of 
CSAC's work could suffer if its resources were spread too thinly. Overall 
investment and (full-time equivalent) enrolments in non-post-secondary 
programs are smaller than in post-secondary programs, and, because of the 
variety of program types and funding sources, the complexity of CSAC's task 
will likely be greater for these programs. 


However, the Board believes that there is as much concern for the quality of 
non-post-secondary programs (from students, faculty, funders and employers) 
as there is for the quality of post-secondary programs. Since these programs 
and the people they serve are as important to Ontario as are the post- 
secondary students and programs, the Board is of the opinion that CSAC 
should concern itself with all publicly-funded post-secondary and non-post- 
secondary programs. The proposed Ontario Adjustment and Training Board 
might assume responsibility for all non-post-secondary programs. CSAC will 
coordinate its efforts with OTAB. 
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The Board believes that it is not necessary, and that it might be counter- 
productive, to require that CSAC standards apply to programs which colleges 
offer under contract to private sector purchasers. Colleges should be free to 
negotiate customized training agreements with private purchasers on terms 
which are mutually agreeable. (Under certain circumstances, such programs 
may be considered by CSAC for accreditation.) Private purchasers also would 
be free to purchase programs which have been accredited by CSAC and, 
therefore, these privately-sponsored students would be eligible to receive the 
same credential awarded to provincially-funded students. 


The inclusion of publicly-funded, non-post-secondary programs significantly 
increases the number of programs under CSAC's jurisdiction. With limited 
resources, CSAC will have to establish priorities and schedule its work over a 
number of years. The questions of priorities and scheduling are examined in 
Section IV of this paper. 


| Do you agree with the Establishment Board's proposal that CSAC 
should have jurisdiction over all publicly-funded programs? 
Why or why not? 
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Section II: Functions of CSAC 


CSAC is intended to have two main functions: (i) developing and setting 
standards for the programs under its jurisdiction which will apply system- 
wide, and (ii) reviewing and accrediting programs to ensure that the 
standards are appropriate and that they are being attained by students. Each 
of these is discussed in this section. 


Included in the broad framework set out in Vision 2000 is the principle that 
CSAC's standards be as enabling as possible. The intent has always been to 
avoid, wherever possible, unnecessary intrusion into college decision-making 
regarding specific curriculum and delivery. Rather, the goal of establishing 
system-wide outcome standards is to attempt, on a province-wide basis, to 
ensure similar outcomes for similar college programs, including 
vocationally-specific and generic skill outcomes and general education 
content, and to assure the public that these standards are being attained. The 
Establishment Board reaffirms this view and is proposing in this discussion 
paper that the colleges continue to meet local needs through their 
responsibilities for curriculum and delivery of system-wide outcomes. 


Section II.A. outlines the Establishment Board's discussions and conclusions: 


(1) regarding information to be included in program standards 
documents; 


(2) that most, but not all, program-specific vocational outcomes, will be 
determined by program councils while leaving some to be set by 
individual colleges to meet local needs; 


_ (3) that minimum generic skill outcomes will be developed for each 
level and type of credential; and 


(4) that students will be exposed to general education according to a set 
proportion of program hours. 


3 By appropriate we mean not only that standards are current, but also that they are relevant 
and meet the needs of both employers and students. In addition, assessing the 
appropriateness of outcomes may include an evaluation of whether particular learning 
outcomes are given sufficient emphasis or whether, for example, employers or graduates see 
the need for greater or lesser emphasis among the approved learning outcomes. By 
attainment, we mean not the assessment of individual students, but that through program 
review, a variety of indicators will assure that students, as a group, are attaining the 
ait and also provide information on how outcomes are achieved and measured by 
colleges. 
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II.A. Program Standards 


II.A.1. Program Standards Documents 


For each similar Program under CSAC's jurisdiction, a system-wide program 
standards document will be developed. Given the large number of programs 
in the system and the financial constraints that CSAC must operate within, 


the Establishment Board proposes that this process take place over a period of 
_ time (see Section IV.A. - Priorities). 


Included within the program standards document will be statements of 
system-wide vocational learning outcomes to be achieved by students on 
graduation from the program, the minimum generic skill outcomes which 
have been established for this level and type of credential, and any generic 
skill outcomes beyond the minimums established. The document will also 
indicate the amount of general education which has been specified for this 
level of credential. 


As well, program standards documents will specify system-wide program 
eligibility requirements, that is, the level of secondary school credits required 
(basic or general) and any specific secondary school courses, for example, grade 
12 English or grade 10 mathematics, needed for admission to the program. 
The intention here is to ensure that such eligibility requirements will be 
reasonable and consistent on a system-wide basis for all similar programs. 


In addition to the system-wide program standards document, each college 
will develop and submit to CSAC any additional learning outcomes which 
have been established to meet local needs. Each college will also indicate how 
the general education requirement will be met and how generic skill 
outcomes will be incorporated into the program, whether taught in discrete 
courses or integrated with vocational learning outcomes. 


Once a program standards documents has been approved, it will be used in 
the following ways: 


* by college faculty and program administrators to plan the delivery of the 
program and to develop detailed curriculum, 


* as a basis.for review of college programs. 


CSAC Discussion Paper March 1992 Page 9 


In addition, program standards documents may be used: | 


¢ by colleges and secondary schools to provide better information and 
guidance to students about college programs (including the admission 
requirements for specific programs), 


° by colleges as a basis for negotiating laddering or articulation 
arrangements with schools and universities, 


¢ by those involved in prior learning assessment, as a benchmark against 
which an individual's skills and knowledge can be assessed, 


¢ by individual employers, unions or organizations to plan their own 
training programs to augment the skills and knowledge of college 
graduates 


CSAC will monitor programs and, when necessary, consider changes to the 
program standards document prior to program review (for example, should 
changes in technology require an immediate change in statements of learning 
outcomes). We are distinguishing here between particular changes that may 
be required on an ongoing basis as opposed to the changes that may be the 
result of recommendations following the normal cycle of program review. 


II.A.2. Vocational Outcomes 


Both vocational outcomes and generic skill outcomes will be expressed as 
learning outcomes in such a way that (i) they can be meaningfully reviewed 
with respect to their appropriateness and achievement, and (ii) they do not 
dictate specific curricula and/or methods of delivery to the individual 
colleges. Learning outcomes will be expressed as statements of knowledge, 
skills, and abilities students will possess on graduation from a specific 
program.4 


In some emerging areas, such as environmental studies, and international 
business, an initial priority for CSAC will be to identify those programs with 
essentially similar objectives. This rationalization of program titles is 
consistent with CSAC's objective of setting common standards for essentially 
similar programs. It should be done jointly by CSAC and MCU in 
consultation with the appropriate college operating group (e.g. Heads of 
Business). 


4 In many sections of this paper, the term 'outcomes' is used synonymously with ‘learning 
outcomes’. 
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Any generic skill outcomes deemed appropriate for a specific program which 
go beyond the minimums established for each level of credential will be 
considered to be vocationally-specific. For example, if numeracy is defined for 
a particular level of credential as knowledge of basic algebra and a program 


requires a student to be familiar with calculus, the study of calculus would be 
considered vocationally-specific. 


II.A.3. Generic Skill Outcomes 


In the Vision 2000 process, concerns were raised by both internal and external 
college stakeholders that college curricula tend to be too narrowly focused on 
specific skills training. Employers ask that 


college graduates have the ability to learn additional skills, to work with others, to 
solve problems and to communicate clearly. These skills are becoming more valued 
because both the speed and nature of economic change make adaptability a 
requirement. From the perspective of employers, a focus on generic skills assists them to 
use both capital and human resources more flexibly in adjusting to the impact of 
technological change and competitive pressure. (p.36) 


In response to these concerns, Vision 2000 recommended a substantial 
increase in the amount of both general education and generic skills to be 
included in all programs leading to a college credential. The Establishment 
Board concurs with this recommendation and its proposal for defining and 
implementing generic skills follows. 


In the context of Vision 2000 and later documents, generic skills have come to 
be understood as "particular life skills essential for both personal and career 
success." These practical, portable skills are defined here as: 


° literacy 

¢ numeracy 

° computer literacy 

* interpersonal skills, and 

° analytical skills including critical thinking and problem-solving. 


The Establishment Board discussed concerns over the inclusion of 
interpersonal skills. The discussion suggested that there may be some 
particular difficulties in defining appropriate interpersonal skills and in 
assessing their attainment. 


Should interpersonal skills be included as one of the generic skills? 
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The CSAC Board will establish minimum outcomes for each generic skill. 
These outcomes will differ by level and type of credential and, where 
necessary, within a level of credential; for example, minimum outcomes set 
for one year post-secondary programs for learners with high school courses at 
the basic level may differ from those established for one year programs where 
learners have credits predominantly at the general level. Organizations such 
as the Business and Technician Education Council (BTEC) in the United 
Kingdom have defined generic, or common, skills under seven headings and 
list performance criteria for each which become increasingly more 
sophisticated for each level of credential.° 


For the most part, the Establishment Board envisions that these generic skills 
could largely be taught as integrated components of all college courses. 
Where appropriate, some generic skills may be offered as discrete courses, 
either within programs or college-wide. Such courses might include 
computer literacy, mathematics, and business/technical communications. 
The Board discussed various methods of measuring student attainment of 
generic skills and concluded that students will demonstrate their mastery of 
these skills within their program rather than though any system-wide 
student assessment instrument. 


Since generic skills will be expressed in terms of outcomes that must be 
demonstrated by students, there is no need to further specify that, for 
example, a set number of program hours must be dedicated to generic skills. 
How program review will examine the attainment and appropriateness of 
generic skills is discussed in Section II.B.3 below. 


II.A.4. General Education 


In presenting a revised mandate for Ontario's colleges, Vision 2000 stated: 


Education should give people the opportunity to develop the skills and knowledge they 
need to adapt to and make a constructive contribution to the world in which they live. 
Education should enhance students’ choices and opportunities, and promote the 
development of individual potential. It should also assist learners in developing their 
commitment to social responsibility and care for the communities in which they live, and 
respect for cultural integrity and self-determination of those whose language and 
traditions may be different from their own. (p. 27) 


... the communications revolution has expanded the horizons of citizenship so that people 
can and should feel part of local, national and international debates on issues that affect 
them, their families and their futures -- issues such as poverty, the environment, the 
Canadian constitution or political change in other parts of the world. To participate 
actively, they should be aware of the background and context of current events and issues. 
Helping people to be good citizens, as well as productive workers with marketable skills, 
should be part of the educational experience at a college. (p. 36) 


More information on the BTEC model, including some examples, may be found in Appendix D. 
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three important goals. First, general education may enhance citizenship 
which includes increasing college graduates’ understanding of themselves 
and their society, their ability to function effectively as members of society, 
and their ability to formulate and to achieve personal goals. Second, while it 
is important to understand the distinction between general education and 
generic skills, it is also important to recognize the link between them. The 
subject matter of general education provides an important context in which 
generic skills may be developed. A strong argument can be made, for 
example, that learners' language and reasoning skills are developed through 
the study of literature, philosophy, sociology, etc. Third, the acquisition of 
general education may assist learners in pursuing options for lifelong 
learning, for example, by providing basic knowledge and potential credit 
toward a (university) degree program. 


In describing general education, the Board, however, wanted to avoid too 
narrow a definition of general education, namely to restrict it to solely the 
study of traditional liberal arts disciplines. While the Board considers these 
disciplines to be a necessary component of a college education, they concluded 
that general education can reasonably include other elements as well. 


Having agreed with Vision 2000's recommendation that there be a significant 
increase in the amount of general education in college programs, the 
Establishment Board debated different ways to implement it. One proposal 
discussed would have provided for a general education council to develop 
specific general education outcomes. However, the Board concluded that 
developing specific outcomes would be both impractical and undesirable. In 
particular, agreement on specific general education outcomes would be 
difficult to achieve; such outcomes would likely have to be stated in fairly 
narrow terms and, thus, dictate curriculum to the colleges; and, to be 
effective, general education requires a commitment from a college and its 
faculty. Such commitment would be more likely to be achieved if major 
decisions about general education offerings were left to colleges. 


Given that the goals of general education are rather broad, the Board felt that 
they should be expressed not as outcomes but rather as exposure to a variety 
of subjects outside a person's program of study. The Establishment Board 
recognizes that expressing general education goals as exposure rather than 
outcomes is a departure from the basic orientation of Vision 2000. However, 
the Board concluded that the commitment to general education requires a 
clear and enforceable rule which can enable colleges to provide learners with 
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the broadest possible exposure, and allow the greatest opportunity for student 
choice. Therefore, the Board does not wish to require colleges to identify 
specific outcomes given that the benefits of general education are many and 
varied for individual learners. 


In the context of the desire to employ a broad notion of general education and 
to measure this through exposure, rather than specific outcomes, the 
Establishment Board proposes that CSAC adopt the following: 


(1) General education should constitute 30% of program hours for all college 
programs within ten years following the establishment of CSAC, and that 


(a) an interim goal of 20% of program hours should be attained within 
five years of CSAC's establishment; and 


(b) after the initial five years, CSAC should develop specific plans for 
attaining the 30% goal. 


(2) In implementing the 20% interim target, 


(a) half (i.e. 10% of program hours) should be achieved through exposure 
to the traditional liberal arts disciplines ; and 


(b) half (i.e. the remaining 10%) may be achieved through 
(i) additional liberal arts courses 


(ii) other offerings outside the specific vocational subjects included in 
student's program: 


¢ existing courses from other programs; and/or 


e new offerings which present either content from other 
programs redesigned and presented for the non-specialist or 
interdisciplinary offerings which may combine content from 
various subject areas. 


The traditional liberal arts disciplines are generally understood to include 
natural sciences (such as biology, chemistry, physics), social sciences (such as 
sociology, psychology, economics, political science) and humanities (such as 
philosophy, history, literature). None of these disciplines is the focus of an 
existing college program (except the General Arts and Sciences program). 
Under the Board's proposal, at least ten percent of program hours would be 
devoted to one or more of these areas. Whether delivered as single subjects 
or as part of a survey course, the focus of this requirement is exposure to the 
traditional liberal arts disciplines. However, where, for example, physics is a 
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required course for a technology 
considered to be fulfilling part o 
students in that program. 


program, the physics course would not be 
f the general education requirement for the 


Offerings outside the student's program could include courses from existing 
programs. For example, a technology student might undertake courses such 
as business administration, accounting or small group behavior. 
Alternatively, colleges might offer courses that are related to existing 
programs but which are redesigned and delivered to students from across the 
college, for example, technological literacy for non-technologists, business 
practices for non-business students, social issues and policies for non-social 
service students. As well, inter-disciplinary courses in areas such as 
environmental studies, applications of new technologies, would also be 
considered, as would theme-based courses such as marriage and the family or 
occupational and community health’. 


The Establishment Board considered the issue of whether the general 
education requirement could be met by exposure to either a single area in 
some depth or a requirement for students to be exposed to a broad range of 
subject areas. The Board concluded that neither CSAC nor individual 
colleges should specify either depth or breadth. Rather, students should have 
the opportunity, to the greatest extent possible, to be exposed to the subjects of 
their choice. 


The Establishment Board feels its proposal has a number of benefits. First, the 
broad range of offerings (including both traditional and new) will allow 
colleges greater flexibility than would have resulted from the imposition of a 
rule specifying only the traditional liberal arts disciplines. This flexibility 
provides an important opportunity for colleges to utilize existing faculty in 
new ways and, therefore, might be less disruptive in the short term than a 
more prescriptive approach. A second benefit of the proposal is that colleges 
may be able to offer students considerable choice in how their general 
education requirement is met, including some exposure to other occupational 
areas. A third important benefit is that colleges may develop new courses 
which are imaginative in both their content and delivery, and are, therefore, 
responsive to learners’ needs. 


Do you agree with the Establishment Board's proposal to define general 
education as including, but not limited to, the traditional liberal arts 
subjects? Why or why not? 


Given this definition, do you agree with the targets the Establishment 
Board is proposing for exposure to general education? 
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In addition, the Establishment Board recognized that one difficulty in 
specifying a rule in the form of a percentage is the great variation of program 
hours. Table A in Appendix C shows the differences in ‘teaching’ hours for 
the twenty largest post-secondary programs offered by the colleges; 17 of 20 are 
offered by more than ten colleges. As is apparent, the variation within many 
programs is significant.6 In addition, however, there is also considerable 
variation among programs defined as, for example, 2 year programs. 


Among the implications of these variations is that exposure to general 
education would exhibit similar variation. For example, being exposed to 
20% of 1000 teaching hours is quite different from 20% of 1800 hours, each of 
which would be the requirement for Early Childhood Education programs at 
different colleges. Some may consider this variation to be acceptable; others 
may consider it too great for what are otherwise similar programs. 


To the extent that the proposal in its most general form would not address 
the variations in program hours either within a program (i.e. the same 
program at different colleges) or between programs (i.e. different programs of 
the same general length), two alternative means to implement the proposal 
might be considered. 


In one version, the proportion could be applied - on a program by program 
basis - to the mean number of program hours or, alternatively, to a prescribed 
minimum number of program hours. This would address the variations 
within each program, but would not address those differences between 
programs. 


_ In the second version, the between program variance could be addressed by 
applying the proportion to the mean (or minimum) program hours for all 
programs offering the credential (e.g. for all two-year diplomas). 


In either version, the amount of general education would be expressed as a 
number of hours, though this exposure would have been calculated by 
applying a percentage rule. 


What advice should the Establishment Board offer about how to | 
measure exposure? 


6 There is no assumption that the differences result in significantly different learning outcomes 
for students. Given that each program is defined as sharing the same basic objectives, it is 
quite possible that the variation represents differences only in delivery. Presumably, the 
process of developing system-wide standards will yield information on the extent to which 
the variation reflects differences in delivery rather than outcomes. 


a _———— 
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The above guideline for general education is, for the most part, intended to 
pertain only to post-secondary programs. Many adult training programs, 
particularly some access programs such as OBS (Ontario Basic Skills) and ESL 
(English as a Second Language), focus almost exclusively on generic skills, 
which are often taught in the context of general education subjects. For 
example, in ESL, a unit on government may be used to teach students new 
vocabulary and may include lessons on verb forms and syntax, as well as 
providing students with knowledge of Canadian government institutions 
and processes. At the same time, in some other adult training programs the 
inclusion of general education may be deemed by CSAC to be neither practical 
nor appropriate given the duration of the program. However, the Board is of 
the view that how to include general education in skills-oriented adult 
training programs should be investigated. 


II.B. Program Review 


To ensure that system-wide standards are being maintained and that 
programs leading to a college credential continue to deliver the learning 
outcomes required by those standards, Vision 2000 recommended that "All 
programs leading to a college credential should be subject to regular, system- 
wide program review for the purposes of accreditation.” (p.43) 


Vision 2000 believed that the following additional benefits would result from 
regular, system-wide program review: 


° program objectives would be regularly revised based on information 
Systematically collected from the colleges; 


e the results could provide the basis for initiatives in faculty, staff, and 
curriculum development; 


* an analysis of comparable programs might provide information on the 
efficiency and effectiveness of alternative delivery modes; 
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e the introduction of this process would provide the impetus for the 
delivery of new evaluation measures and instruments, including new 
means of assessing students, tools for gathering input from students, 
employers, faculty and administrators, and means to examine 
instructional processes. (p.45) 


The following describes the process for program review envisaged by the 
CSAC Establishment Board, including the different means of evaluating the 
attainment and appropriateness of vocationally-specific outcomes, the 
attainment of generic skills outcomes and the learner's exposure to general 
education. 


II.B.1. Program Review Framework 


The Establishment Board is proposing that CSAC's program review process be 
designed in a manner which will include the following: 


¢ the process will be collaborative and inclusive, with the involvement of 
internal and external stakeholders, and where necessary, experts 
external to the college system; 


° a variety of indicators of the attainment and appropriateness of the 
intended outcomes will be utilized in evaluating a program; no single 
indicator will be a sufficient measure upon which to evaluate a 
program; 


e CSAC will have the authority to use the results of the program review 
process to determine whether or under what conditions a program 
should continue to be accredited. 

The following stages are proposed for the program review process: 
(1) Program councils will develop an Inventory of Basic Descriptive Program 

Information. This information will be submitted by each college for each 


_program it offers and will include the following: 


e the specific aims and objectives of a program as defined by each college 
including additional college-specific vocational outcomes; 


¢ the length of the program, credential awarded and transfer 
opportunities; 


e the structure of the programs (i.e. subjects, work and clinical experience); 


/ 
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¢ brief descriptions of curriculum 
generic skill and vocationally-s 
the way in which the general 


including the means by which the 
Pecific outcomes will be delivered and 
education requirement will be met; 


¢ the methods of student assessment used in the program; and 


¢ the results of basic monitoring of the program such as admissions, 
attrition, and completion. 


(2) CSAC's program councils will develop a series of common indicators or 
instruments. These should include Surveys of employers, students, 
faculty, and administrators. Information would be collected on both the 
attainment and appropriateness of the learning outcomes established for 


the program as well as suggestions about changes required to ensure the 
relevancy of the standards. 


(3) Colleges will submit a Critical Appraisal of the program under review 
prepared by the local program advisory committee supplemented by 
faculty, students and administration. This document would include 
Statistical information, evaluative comment drawn from several sources 
and indications of action taken to meet problems identified. In Critical 
Appraisals submitted to the Council on National Academic Awards (U.K.) 
the following areas are included: 


° a statement of the institution's internal procedures for monitoring and 
evaluation of the program with relevant reports where it is useful to 
include them; 


* an evaluation of the extent to which the program has fulfilled its aims 
and objectives including critical comments on the attainment of 
program outcomes by students and on methods of student assessment; 


° a report on the changes introduced into the program to improve it, either 
to build on strengths or remedy weaknesses -- for example, on the fulfill- 
ment of any conditions or recommendations from prior program review; 

¢ discussion of data collected through common instruments developed 
by the program councils and data contained in the inventory of basic 
descriptive program information (e.g., progression of student cohorts); 


e information on changes in resource provision affecting the program; 


* proposals for future development. 
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(4) A review panel, appointed by the program council, will review the 
standards documents, the inventory, the common indicators and the 
critical appraisals. 


(5) The review panel will determine if any follow-up is required, either 
additional documentary information or a site visit. If no follow-up is 
necessary, the panel would be able to recommend to the program council 
the continued accreditation of the program. 


(6) The review panel would prepare its report and submit its 
recommendations to the program council and each college for any 
response and discussion prior to a formal decision. Every effort should be 
made to achieve consensus on possible conditions or recommended 
changes. In the event of a formal appeal by a college of an accreditation 
decision, the program council would continue to involve the college and 
the review panel in a collaborative process to resolve their differences. 
However, the program council will have the authority to decide appeals; 
the CSAC Board should not re-adjudicate program review decisions. 


(7) Review panels will recommend one of the following: that the program be 
unconditionally accredited, that it be conditionally accredited, or that 
accreditation be withdrawn from the program. Programs accredited 
without condition will not be reviewed until the next cycle (e.g. five 
years). Those programs accredited with conditions should be re-evaluated 
following a set period (e.g. one or two years) as stipulated by the review 
panel. 


If program standards themselves are found to require revision, the review 
panel could make recommendations to the program council. 


Are there other elements of program review which should be included? 


What information gathered as part of program review should be 
| published? What considerations are there in terms of the Freedom of 
| Information Act? 


In its discussion, the Establishment Board concluded that it would be useful 
to distinguish between some of the elements of program review particularly 
relevant to each of vocationally-specific outcomes, generic skill outcomes, 
and general education. 
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ILB.2. Vocational Skills 


Program-specific vocational outcomes will be reviewed according to the 
process described above. The revi 
the critical appraisal, and, possibly, assessment instruments and samples of 

student work in relation to the outcomes identified in the Program Standards 


II.B.3. Generic Skill Outcomes 


Generic skill outcomes, both those stipulated by CSAC as minimums and 
those specified by program councils as vocationally-specific, will be reviewed 
as part of regular program review using the process described above. 


The Board discussed the use of gateway tests as a means of measuring 
Students' attainment of generic skills. (As they are used in a number of 
jurisdictions, gateway tests are required for graduation or to proceed to further 
education. Such tests measure both skill and knowledge.) The Board 
concluded that these kinds of instruments are not an appropriate means of 
measuring student ability, given the many questions which exist concerning 
(i) test biases, (ii) the validity of testing generic skills apart from a specific 
context, and (iii) their administrative cost. As well, the Board affirmed the 
view that the purpose of CSAC's program review is to evaluate the 
effectiveness of programs, not the performance of individual students. 


In the State of New Jersey, a test of general intellectual skills (known as the 
G.I.S.7), which is markedly different in both content and purpose from 
gateway tests, is given to a sample of students at each post-secondary 
institution in order to measure the effectiveness of the program/college in 
terms of student attainment of certain generic skills. The Board decided that a 
GIS-type instrument might be usefully employed as a measure of one aspect 
of program effectiveness by testing a sample of graduating students for their 
attainment of selected generic skills such as gathering, analyzing and 
presenting information. 


Should there be a province-wide assessment of certain generic skills, the 
purpose of which would be to evaluate program effectiveness, not 
individual students? 


aN description of the New Jersey approach is found in Appendix E. 
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II.B.4. General Education 


As part of program review, general education will be examined to determine 
whether students have successfully completed a quantity of general education 
in accordance with the established rule. 


On a regular basis, each college would submit a Critical Appraisal of its general 
education offerings to a General Education Review Panel evaluating, for 
example, the specific aims and objectives of general education courses at the 
college, the methods of student assessment used and the college's evaluation 
of student attainment. As well, using common instruments, perceptions of 
employers, students, faculty and administration would be gathered. 


II.C. Credentials 


The authority to define non-degree credentials is an important vehicle by 
which CSAC can ensure that its goals are met. While Vision 2000 suggested 
the need for CSAC to redefine credentials offered by the colleges, the 
Establishment Board has named the major elements to be used to do so. 
These elements include the vocational and generic skills outcomes 
(declaration 2) as well as the general education component (declaration 3). 
There remain, however, several unanswered questions. 


As Vision 2000 noted (p.40), college credentials should convey particular 
information and communicate this clearly to employers, students, 
educational institutions and the public as a whole. Many have suggested that 
the current titles of credentials fail to do so, that there is a need to clearly 
differentiate among the certificates and diplomas of different levels and types 
of programs (e.g., most non-post-secondary programs award ‘certificates’, as do 
one year post-secondary programs and post diploma programs). The issue is 
not simply the titles used, but the kind of information they should convey. 
The Establishment Board has not yet discussed this matter in detail. 


However, it may be useful to note that in the legislation establishing the New 
Zealand Qualifications Authority (NZQA), a body responsible for all 
secondary and post-secondary credentials, the government of New Zealand 
mandated the NZQA itself with the responsibility for developing a new 
“framework for qualification” (i.e. the titles of and the relationships between 
the various credentials). 


It can also be noted that in most other jurisdictions, credentials are 


differentiated by the number of credit hours required, with a minimum 
number set for each credential. 
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same credential. 


Some members of the college community would suggest that using program 

hours to define credentials would fail to recognize the importance of defining 
programs in terms of ‘outcomes’ rather than ‘time’, whether measured as 
hours or program duration. Others might suggest that while it may be 
conceptually appropriate to define credentials through program hours, if this 
definition were other than very general, it would have the practical 
consequence (or potential) of intruding into the colleges’ responsibility for 
delivery. 


Therefore, the reader may wish to address the following questions: 


How should CSAC redefine credentials? Should credentials be defined 
by the duration of programs? by the kinds of outcomes expected? by 
other features? and if so how? 


What kind of information should college credentials convey? 


Should credentials distinguish levels (e.g. certificates lead to 
diplomas which lead to ...)? 


Should college credentials differentiate between CSAC-accredited 
programs and those which are not within CSAC’s jurisdiction? 


II.D. Assessment and Remediation 


The Establishment Board considered assigning to CSAC the responsibility for 
assessment and remediation to include monitoring, coordinating and 
evaluating province-wide, post-admission, basic skills assessment and 
remediation for post-secondary students. In its discussion, the Board . 
considered problems faced by the colleges because of the number of 
academically under-prepared students enroling in post-secondary programs, 
the variety of means by which students are being assessed at individual 
colleges and the varied approaches to preparatory and remedial programs. As 
well, the problem of funding for preparatory and remedial programs was 
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discussed. While recognizing the seriousness of this issue, the Board 
concluded that such activity does not logically reside within CSAC's mandate 
to set standards for and accredit college programs. Again, the Board 

reinforced its view that CSAC should not be involved in determining policies 
concerning the assessment of individual students. The Board also felt that 
extending CSAC's mandate, especially in its formative years, might prevent it 
from accomplishing its primary goals. However, the Board is of the view that 
a body other than CSAC, but with representation from appropriate 
stakeholders, should be mandated to examine these issues. 


Are there other functions which the Establishment Board should 
consider for CSAC ? 
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Section III: Structure and Relationships 


This section outlines an organizational structure for 


carrying out the mandate 
and functions of CSAC as described in the previous s Sona 


ections of this paper. 


III.A. Internal Structure 


Standards, and accreditation are subjects of interest to almost everyone in the 
college community. Students, faculty, support staff, administration, 
governors, employers, unions, professional associations, regulatory 
authorities, universities, school boards and community groups each will 
have their unique perspective on these issues. If CSAC is to do the best 
possible job, it must take account of the interests of each of these 
constituencies. Representatives of each must be a part of the CSAC process. 


Both standards, and accreditation have to do with the way in which the 
colleges relate to individuals and organizations outside the college system. 
Consequently, membership on CSAC bodies must reflect a balance of internal 
and external interests. Similarly, the internal representation on CSAC bodies 
must reflect the partnership between management, labour (faculty and 
support staff) and students. 


It is proposed that the composition of the CSAC Board and subsidiary bodies 
conform to the following guidelines. 


(a) Half the membership will be internal to the college system and half external. 


(b) Among the internal members, the number of representatives from 
management will equal the number of representatives from OPSEU 
(faculty and support staff). 


(c) Organizations representing the constituencies (e.g. OPSEU representing 
faculty and support staff) will nominate members to the CSAC Board and 
its subgroups. Appointments to the CSAC Board will be made by the 
-Government of Ontario. Appointments to CSAC subgroups will be made 
by the CSAC Board. 


(d) Appointments to the CSAC Board and its subgroups should be made with 
the objective of reflecting the diversity of Ontario's population. 


(e) In accordance with CSAC's arm's length relationship with government, 
representatives of government and its non-college agencies will be non- 
voting. 
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TABLE ONE: PROPOSED MEMBERSHIP ON THE CSAC BOARD 


Chair 


Internal Constituencies 
College Boards and Administrators 
(e.g. Council of Governors, Council of Presidents 
College Committee on Academic Affairs) 
Ontario Public Service Employees Union 
Faculty and Support Staff 
Students (e.g. Ontario College Student Presidents’ 
Association) 


External Constituencies 
Employer and Industry Groups 

(e.g. Canadian Manufacturers’ Association) 
Labour Organizations 

(e.g. Ontario Federation of Labour) 
Professional Associations 

(e.g. Ontario Association for Engineering 

Technicians and Technologists) 
Regulatory Bodies 

(e.g. College of Nurses of Ontario) 
Universities and School Boards 

(e.g. Council of Ontario Universities) 
Community Groups 


N on-voting Members 


Council of Regents 
Ministry of Colleges and Universities 
Ontario Training and Adjustment Board 


Total 
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CHART ONE: PROPOSED ORGANIZATIONAL STRUCTURE FOR CSAC 


~ 


CSAC BOARD 


' SECRETARIAT SPECIAL PURPOSE 
COMMITTEES 


e.g. Generic Skills 


PROGRAM COUNCILS 


Access (e.g. ESL, OBS) 
Applied Arts 
Business 

Health Sciences 
Hospitality 

Social Services 
Technology 

Trades 


PROGRAM STANDARDS PROGRAM REVIEW 
COMMITTEES PANELS 


(e.g. Chemical Technology) (e.g. Chemical Technology) 


Is the proposed structure of CSAC appropriate to its mandate 
and functions? 
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The CSAC Board 


The major decisions about CSAC's policies, and priorities will be made by the 
CSAC Board. The responsibilities of the Board will include: 


_® approving program standards documents and accreditation decisions; 
e setting policy and priorities, and developing guidelines for 


- general education 
- generic skills 

- program standards 
- program review 

- accreditation and 

- college credentials; 


¢ monitoring the implementation and impact of program standards, 
program review and accreditation; 


e representing CSAC to other organizations (e.g. government, regulatory 
bodies) and to the public (e.g. through publication of program standards, 
accreditation decisions, policies and guidelines). 


The Establishment Board gave some consideration to the appropriate size for 
the CSAC Board. While a small board (10 to 15 members) might be capable of 
reaching decisions more quickly, in order to secure the necessary diversity of 
external stakeholders the Board may have to be larger rather than smaller. 
The 22 member board proposed in Table One might be considered to be a 
moderate sized board. 


Have the appropriate constituencies been identified? 
Are the numbers of representatives appropriate? 


Program Councils 


Vision 2000 recommended the creation of a series of program councils. Each 
program council would assist CSAC in implementing standards, program 
review and accreditation for a cluster of related programs. A program council 
will fashion unique solutions to the unique problems for programs in its 
cluster while seeking consistency in approach among those programs. 
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Program councils would be composed of persons involved with programs or 
occupations represented in the cl 


reasonable limits. 


Are program councils necessary? Is there a more suitable alternative? 


Program Standards Committees 


The work of developing specific Program standards will be the responsibility 
of program standards committees. This work will require the cooperative 
efforts of faculty, program administrators, workers, employers (and others 
where necessary) who have expert knowledge of the specific program and/or 
occupation(s) for which the standard is being developed. Committee 
membership will follow the rule of half internal and half external 
representatives. Representatives will have to be knowledgeable in the 
program and/or the occupations for which the program is preparing learners. 


While, in principle, a separate program standards committee will be needed 
for each program standard, there are many instances of closely related 
programs (such as chemical technician and chemical technology) which 
might be served by a single committee. Each program standards committee 
will report to the program council responsible for the cluster to which the 
program belongs. 


Program Review Panels 


Three to five years after CSAC has approved a program standard, the program 
council will create a program review panel to assess the implementation of 
the standard by colleges. It is expected that review panels will have 4 to 6 
members (some internal and some external) with provision for additional 
members if needed. CSAC will attempt to ensure that those external 
organizations with the greatest stake in the program are represented on the 
review panel. There will need to be conflict of interest rules governing the 
conduct of members with respect to programs at individual colleges with 
which they may have some affiliation. 
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Where site visits are required, the members of the review panel might do the 
visits themselves or they might get other knowledgeable people (including 
CSAC staff) to do the visits. It is expected that site visit teams will have one to 
three members depending on the kind of information they have to collect. 


What should be the size and composition of review panels? 


Special Purpose Committees 


One or more special purpose committees may be required in the initial years 
to assist the CSAC Board to develop specific guidelines, or instruments. For 
example, an ad hoc committee on generic skills might develop a framework 
and learning outcomes for generic skills, and possibly oversee the 
development of an assessment test of generic skills. 


Secretariat 


The secretariat will provide staff support to the CSAC Board and its subsidiary 
bodies. The secretariat might initially consist of a director (who would report 
to the CSAC Board) six professional staff and three support staff. 


III.B. External Relationships 


CSAC's Location with Respect to Government 


CSAC program standards must reflect a balance among the interests of 
colleges, employers, labour, professional groups, regulatory bodies and 
government. To be perceived as being truly representative of the college 
community (broadly defined), CSAC and its subsidiary bodies must be seen to 
operate independent of any single interest. One way of achieving such 
independence would be to establish CSAC as an independent government 
agency (like the Council of Regents or the Ontario Council on University 
Affairs). CSAC would be funded by government but would control its own 
budget, employ its own staff and have clear authority with respect to program 
standards and accreditation. 
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What are the most important considerations which should guide the 
decision about where CSAC should be located? 


Checks and Balances: Relationship with MCU and OTAB 


Given that CSAC has authority to set standards and accredit programs, CSAC 
and the Ministry of Colleges and Universities will have to work out ways of 
coordinating CSAC standards with Ministry funding levels. Similar 
arrangements will have to be made for non-post-secondary programs -- 
presumably, with the proposed Ontario Training and Adjustment Board. 


With respect to funding, CSAC and MCU might interact in the following way: 


(1) In developing program standards CSAC will work with colleges to 
estimate the effect of each new standard on the cost of a program. The 
CSAC Board would not approve a program standard which increases the 
cost of a program unless there is compelling evidence demonstrating: 


° why the specific standard is necessary, and 
¢ why the standard will increase the cost of the program. 


(2) Based on 1 above, CSAC and the Ministry will jointly review the funding 
implications of the program standard. CSAC program standards may 
Provide useful evidence in support of Ministry global funding 
submissions to Treasury Board. 


(3) In the event that insufficient funding is available to the college system to 
implement the changes required by a program standard, a number of 
responses are possible: 


° CSAC could adjust the program standard to match the existing funding 
level. 

e CSAC could approve the standard, leaving colleges to find the 
necessary resources from other sources, or to stop offering the 
program. 

¢ MCU might introduce an enrolment quota for the program 
while increasing the per-student funding. 
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In the event that CSAC reports to the Council of Regents, the above protocol 
would involve the Council of Regents in any significant discussions or 
agreements with the Ministry or OTAB. 


A related issue is the relationship between the accreditation status and the 
Ministry's funding of a program. While the final decision with respect to 
continued funding would rest with the Minister, the expectation would be 
that a program that fails to receive either conditional or unconditional CSAC 
accreditation would cease to receive provincial funding. 


Consultation with respect to French Language and Native Programs 


College programs targeted to specific populations (e.g. Francophones, Natives, 
the disabled) will present a special challenge to CSAC. Programs with cultural 
references (e.g. Early Childhood Education, Journalism) may have 
significantly different objectives for Francophones or Natives than for - 
Anglophones. The CSAC Board will need special advice with respect to 
programs targeted to specific populations and with respect to the impact of all 
program standards and program review processes on these populations. 
CSAC will work closely with the Advisory Council on Francophone Affairs 
and the Native Education Council to develop appropriate processes to ensure 
that the unique needs of these two populations are met. 
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Section IV: Implementation and Costs 


Given the number and range of programs under CSAC’s jurisdiction, CSAC 


will have to set priorities and phase its work of developing program 
standards over several years. 


 IV.A. Priorities 


CSAC might direct its resources initially to developing standards for: 


(1) those large programs which: 
¢ involve the greatest expenditure of public funds, 
¢ are offered at the largest number of colleges, and/or 
e enrol the most students; 


(2) programs for which there is an exceptional need or benefit or for which 
there is a readiness and desire among stakeholders; 


(3) some small programs. (CSAC will want to send the message that | 
quality is important in small and large programs.) 


Selecting the largest programs will ensure that expenditures on program 
standards and review affect the greatest number of college system 
stakeholders. The 75 college programs which receive the greatest provincial 
and federal funding account for approximately 75% of total enrolment and 
funding in college system full-time programs. If CSAC completes 15 program 
standards in each of the first five years, selecting mostly large programs, 
within five years, the majority of college students. would be in programs 
where CSAC had approved standards. . 


The involvement of CSAC in standards setting and program review for the 
former MSD and MOE programs will require close coordination with OTAB. 
Among non-post-secondary programs, apprenticeship programs are a special 
case and will likely require arrangements which differ from other college 
programs. Apprenticeship programs are different because: 


¢ only a portion of apprentice training is done in colleges, 
e standards already exist for many apprentice programs and 


° apprentice training comes under the authority of the Apprenticeship 
and Tradesmans' Qualifications Act. 


What considerations other than the enrolments, funding and number 
of colleges offering a program should guide CSAC in selecting programs 
or the development of standards? 
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IV.B. Staging 


Table Two shows one possible staging for the development of program 
standards and for the implementation of program review procedures. -In 
developing the table, a number of assumptions have been made. 


(1) CSAC will commence operation in 1993/4. 


(2) Review of college programs will be most effective, and most 
consistent with CSAC's mandate, if it is done after colleges have 
made any changes to their programs to meet CSAC program 
standards. 


(3) It will probably take one year to develop and validate a CSAC 
program standard and an additional two to five years for colleges to 


implement the new standards and to graduate their first class of 
students from the revised programs. 


TABLE TWO: PHASING OF STANDARDS AN PROGRAM REVIEW 


| YEAR PROGRAM STANDARDS PROGRAM REVIEW 


1993/4 Standard Setting: 15 Programs 

1994/5 - 

1995/6 Instrument Development 

1996/7 & Pilot Projects 

1997/8 * 

1998/9 Standard Updating: 15 Programs | Program Review: 15 Programs 
| 1999/2000 3 oh i 
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2000/2001 


IV.C. Pilot Projects 


In order to facilitate the implementation of CSAC, the government has 
provided funds for pilot projects related to provincial program standards, 
program review, general education and generic skills. 
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Phase One Projects: 


on program review, are intended to assist the Establishment Board and 
subsequently CSAC, by collecting comprehensive, system-wide information. 
The purpose of the general education Project is to develop an inventory of 
policies and practices regarding the delivery of general education, as it has 
been defined in this paper, in the colleges including a description and analysis 
of the various ways in which general education is delivered in college 
programs. An inventory of policies, practices and specific 
methodologies/tools for program review will be developed through the 
second project. Also included will be an analysis of the current approaches to 
program review in the system. Both projects will submit an interim report in 
May 1992 and a final report in September 1992. 


Over the past two years, a working group of Early Childhood Education 
coordinators has developed a set of professional competencies for the ECE 
program. A third pilot project will build on the work already done. Since the 
standards which have been developed focus on vocational skills, there is a 
need to integrate generic skill outcomes. There is also a need to involve the 
external community formally in the process of developing standards. Of 
further interest will be a discussion of the process envisioned for program 

. Teview including indicators which might be used to determine the extent to 
which the standards are being met. 


The fourth project will focus on program review tools for nursing programs. 
For many years, nursing programs have operated within the framework of 
provincial program standards. These standards have recently been revised. 
As well, in the past, the review of these programs has been conducted by the 
College of Nurses. This function will now be taken over by CSAC. Therefore, 
as colleges move to implement the new standards, new assessment tools will 
be needed. This project will support the development of such tools. 


Phase Two Projects: 


Phase Two projects will be undertaken in the 1992/93 fiscal year and will 
provide funding in two areas. In the first, colleges will be asked to submit ; 
proposals related to developing system-wide outcomes. Six to ten projects in 
different program areas will be chosen. The purpose of these projects will be 
to facilitate the development of standards for college programs under CSAC's 
jurisdiction and to provide useful feedback on the process of standards 
development to CSAC during its formative period. The second type of 
project will respond to the anticipated need for professional development 
related to the creation of outcome standards, program review, or the delivery 
of general education or generic skills. | 
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IV.D. Notes on Costs 

There are likely to be a variety of costs associated with CSAC, not all of which 
can be specified in advance. Two major categories, however, can be 
identified: one-time costs, and ongoing costs. 


The one time costs of CSAC, including those associated with its ‘start-up’, 
consist of items such as: 


e development of a credentials framework (including a generic skills 
framework and generic skills outcomes) 


* initial development of program standards documents 
* initial development of program review instruments 


* one-time administration costs (e.g. information, initial staffing, 
equipment). 


The ongoing costs of CSAC include: 


* ongoing administrative costs (including staff, offices, travel, meetings, 
liaison with MCU, COR, OTAB, colleges, etc.) 


* per diem payments for participants, where required (e.g. review panels) 


periodic revision to standards documents.and program review 
instruments 


° program review (e.g. administering instruments such as employer 
surveys and GIS-type instruments). 


Each of these areas will need to be developed in some detail prior to CSAC’s 
implementation. In addition, there may be costs to the colleges as a result of 
CSAC’s operation.’ First and foremost among these is the possible increased 
cost of program delivery. Specifically, to the extent that programs may be 
lengthened, costs would consequently increase. 


8 For example, under the program review process, colleges would be required to provide program 
information and to develop a ‘critical appraisal’ of the program. It can be assumed that 
many, if not all, of these activities are currently the practice in the colleges, though not all 
are used to evaluate each program. Therefore, while there are indeed likely to be costs to the 
colleges for program review, not all of these are ‘new’ costs and, in the absence to 
comprehensive data, it is difficult to estimate the additional expenditures necessary. 
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In reflecting on the current situation of financial restraint, the Establishment 
Board considered the implications of a Significant increase in general 
education. There was a general agreement that simply adding general 
education to existing program hours was not feasible. 


An alternative to simply adding general education hours is to substitute 
general education for some of the hours now devoted to vocationally-specific 
and generic skill development. While it was unclear how much latitude 
exists to do so without jeopardizing the attainment of vocational and generic 
skills, it was suggested that this kind of Stipulation - i.e. that increases in 
general education must be accomplished within existing program hours - was 
necessary to protect the overall financial viability of the colleges in the 
current fiscal environment. Doing so, however, would appear to assume 
either that a significant portion of vocational outcomes could be eliminated 
from the program or that vocational and generic outcomes could be delivered 
more efficiently. The Establishment Board’s discussion also acknowledged 
the possible disruption to existing faculty of reducing vocational outcomes in 
favour of general education, a reality which provided a further rationale for 
phasing in general education over a sufficient period of time to gain the 
benefits of faculty retirements and new hirings. An additional possibility 
available to address these concerns would find a portion of the increase in 
general education achieved through current hours and a portion provided 
through an increase in hours. This approach would necessitate some new 
funding. : 


Three scenarios are offered in the data presented in Table B (found in 
Appendix C) showing the effects of implementing a substantial increase in 
general education hours through (i) no increase in program hours, (ii) all 
new program hours and (iii) half new hours and half existing hours. Under 
the (conservative) assumption that only 5% of current program hours would 
conform to the Establishment Board's definition of general education (the 
Establishment Board expects more comprehensive data on existing offerings 
to be provided through the general education pilot project, currently under 
way), the table provides information on the increases from 5% to 20%, from 
20% to 30% and from 5% to 30%. 


The table lists nine assumptions concerning both program hours and costs 
associated with their increase. The increases in annual funding estimated 
under each of the scenarios are found at the bottom of the table. As shown in 
the table, the cost of increasing from 5% to 20% would be approximately $84M 
if the entire increase were to take the form of additional program hours; and 
approximately $38M if one-half of the required hours are additional and one 
half are derived from existing program hours. The cost of an increase from 
20% to 30% - the second stage in the Establishment Board proposal ; would be 
an additional $77M adding all new hours and $28M if half the additional 
hours are ‘new’. Finally, the costs of the entire increase, from 5% to 30%, are 
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shown. With the entire increase simply added on to existing program hours, 
a cost of approximately $161M might be incurred, compared to $66M if only 
half the required increase were achieved through additional program hours. 


It should be noted that these scenarios assume current methods of delivery. 
However, there may be some scope for achieving cost saving through 
alternative methods of delivery in vocationally-specific as well as general 
education courses. 


IV.E. Conclusion 


The challenge and the changes proposed by Vision 2000 and by this discussion 
paper are considerable. Ontario needs workers who are highly skilled, 
knowledgeable and flexible problem solvers; and Ontario needs citizens who 
are broadly informed about their complex and changing world. Ontario's 
colleges have a pivotal role to play in serving the workers and residents of the 
province -- a role which will demand excellence from all participants in the 
system. 


We believe that the proposed College Standards and Accreditation Council 
can be a focal point for fostering excellence in programming in Ontario 
colleges. If CSAC is to succeed, it will be because, collectively, we work for its 
success. 


Let us begin by ensuring that the CSAC we create is a CSAC which will meet 


the needs of both internal and external constituencies to our college system. 
Toward this end, we welcome your responses to this paper. 
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Section V: Questions for Responding to Discussion Paper 


Section I: Mandate of CSAC 


1. 


pls 


Are there other matters which should be included in CSAC's mandate? 


Do you agree with the Establishment Board's proposal that CSAC should 
have jurisdiction over all publicly-funded programs? Why or why not? 


Section II: Functions of CSAC 


3. 


4. 


Should interpersonal skills be included as one of the generic skills? 


Do you agree with the Establishment Board's proposal to define general 
education as including, but not limited to, the traditional liberal arts - 
subjects? Why or why not? 


. Given this definition, do you agree with the targets the Establishment 


Board is proposing for exposure to general education? 


. What advice should the Establishment Board offer about how to measure 


exposure to general education? 


- Are there other elements of program review which should be included? 


. What information gathered as part of program review should be 


published? What considerations are there in terms of Freedom of 
Information Act? 


. Should there be a province-wide assessment of certain generic skills, the 


purpose of which would be to evaluate program effectiveness, not 
individual students? 


10. How should CSAC redefine credentials? Should credentials be defined by 


the duration of programs? by the kinds of outcomes expected? by other 
features? and if so, how? 


11. What kind of information should college credentials convey? 


12. Should credentials distinguish levels (e.g. certificates lead to diplomas 


which lead to ...?) 
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13. Should college credentials differentiate between CSAC-accredited 
programs and those which are not within CSAC's jurisdiction? 


14. Are there other functions apart from standards and program review and 
accreditation which the Establishment Board should consider for CSAC? 

Section III: Structure and Relationships 

15. Are the guidelines presented for membership (governing CSAC's internal 
structure) appropriate? Are there other general guidelines which should 


apply to the creation of the CSAC Board and its subgroups? - 


16. Is the proposed structure of CSAC appropriate to its mandate and 
functions? 


17. Have the appropriate constituencies been identified? 
Are the number of representatives appropriate? 


18. Are program councils necessary? Is there a more suitable alternative? 

19. What should be the size and composition of review panels? 

20. What are the most important considerations which should guide the 
decision about where CSAC should be located’? 

Section IV: Implementation and Costs 

21. What considerations other than the enrolments, funding and number of 


colleges offering a program should guide CSAC in selecting programs for 
the development of standards? 
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Appendix A: Members of CSAC Establishment Board 


Co-Chairs: 


Richard Johnston 
Chair 
Ontario Council of Regents 


Members: 


Ralph Benson 
Assistant Deputy Minister 
Ministry of Colleges and Universities 


Eileen Burrows 
President, Local 558 
Ontario Public Service Employees Union 


Carol Gooding 
Chair, Board of Governors 
Sheridan College 


Brian Henderson* 
Director, Education and Accreditation 
Canadian Dental Association 


Cathy Henderson 
Vice President, Academic 
Sheridan College 


Bill Kuehnbaum 
Vice-President 
Ontario Public Services 
Employees Union 


* Resigned January 1992 


Bernard Shapiro 
Deputy Minister 
Colleges and Universities 


Keith McIntyre 


Chair, Council of Presidents 
President, Mohawk College 


Penny Moss* 
Ontario Council of Regents 


Bill Summers 

Manager, Program Services 
Ministry of Colleges and 
Universities 


Ross Traub 

Department of MECA 

Ontario Institute for Studies in 
Education 


Jim Turk 
Director of Education 
Ontario Federation of Labour 


Maurice Yeates 

Executive Director 

Ontario Council on Graduate 
Studies 
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Appendix B: CSAC Establishment Board 


Terms of Reference 


PREAMBLE 


In 1988, at the request of the Minister of Colleges and Universities, the 
Council of Regents initiated a comprehensive, consultative review of the 
mandate of Ontario's colleges of applied arts and technology. The final report 
of this review, titled Vision 2000: Quality and Opportunity, included, among 
its recommendations, the following. 


There should be a significant increase in the generic skills and general 
education content of programs leading to a college credential to ensure 
an equivalence of learning outcomes between these components and 
specific occupational skills. (Recommendation 2) 


There should be system-wide standards for all programs leading to a 
college credential. Such standards must focus on the learning 
outcomes expected of graduates from a program. (Recommendation 3) 


All programs leading to a college credential should be subject to 
regular, system-wide program review for the purpose of accreditation. 
(Recommendation 4) 


A college Standards and Accreditation Council (CSAC) should be 
established, with participation of internal and external stakeholders 
and with executive authority in the areas of system-wide program 
standards, review and accreditation. (Recommendation 5) 


On April 29th, the 1991 Ontario budget was presented to the Legislature. It 
noted in part that "education, in all its dimensions, must be at the centre of 
our efforts to achieve sustainable prosperity," and announced three new 
initiatives deriving from the recommendations of Vision 2000. The first of 
these three initiatives was the establishment of a College Standards and 
Accreditation Council. 
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PURPOSE 


The CSAC Establishment Board has been created for the purpose of 
recommending to the Minister the mandate, structure, nature, relationships 


and any other aspects germane to the establishment of a College Standards 
and Accreditation Council. 


MEMBERSHIP ~ 
The members of the CSAC Establishment Board will be as follows: 


- Chair, Ontario Council of Regents 

- Deputy Minister, Ministry of Colleges and Universities 

- Council of Regents (two additional members) 

- Ministry of Colleges and Universities (two additional members) 

- Ontario Public Service Employees’ Union (two members) 

- Council of Presidents (two members: one college president and 
one representative of the College Committee on Academic 
Affairs) 

- Council of Governors (one member) 

- Ontario Community College Student Presidents’ Association 
(one member) 

- Council of Ontario Universities (one member) 

- Two external experts in the field of standards and accreditation 


The CSAC Establishment Board will be co-chaired by the Chair of the Ontario 
Council of Regents and the Deputy Minister of Colleges and Universities. 


STAFF 


The Acting Manager, Policy and Research, of the Council of Regents will 
serve as secretary to the board. 


A secretariat of up to four full time-equivalent professional staff will be made 
available as needed. 


CSAC Discussion Paper March 1992 Page 43 


OBJECTIVES 


Consistent with purpose of the CSAC Establishment Board, and building on 
the consensus achieved in the Vision 2000 process, the objectives of the Board 
are: 


1. _ to identify models for the structure and function of a College Standards 
and Accreditation Council, and to evaluate the models; 


2. to ascertain the degree of consensus among the college system 
stakeholders (staff, students, employers, unions and _ professional 
bodies, the university and secondary sector) concerning the various 
CSAC models; 


3. to oversee evaluative research and/or pilot projects as appropriate 
concerning program standards, program review, and general education 
and generic skills; 


4. to recommend to the Minister of Colleges and Universities the terms 
of reference for the College Standards and Accreditation Council 
including: 


(a) Mission: Purpose, goals, objectives, and/or governing principles 
for CSAC; 


(b) _ Legislative / regulatory framework: the appropriate (changes to) 
legislation, regulations and/or policy for the establishment of 
CSAC; 


(c) Structure and relationships: Central Council, subsidiary bodies 
(possibly program councils), standing committees, their 
relationships to each other and to other bodies (e.g. MCU, COR, 
boards of governors, ACAATO, professional and regulatory 
bodies); 


(d) | Membership: constituencies to be represented, method of 
appointment, terms and conditions of membership; 


(e) Jurisdiction: the domain of college programs within the purview 
of CSAC and any subsidiary bodies; 


(f) Functional responsibilities: (e.g. establishing and maintaining 


outcome standards, conducting program review, accrediting 
programs); 
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(g) Authority: areas over which CSAC would have executive 
authority, and areas in which it would play an advisory role, 
(and to whom it will provide such advice); 


(h) Accountability: reporting requirements by CSAC (nature, 


frequency, and to whom) and to CSAC (nature, frequency and by 
whom); 


(i) Processes: (to the extent appropriate for terms of reference) for 
the various functional responsibilities and the various programs 
within CSAC's jurisdiction, including appeal processes; 


(j) | Funding: source and amount of funding including grants, fees (if 
any) to institutions, students, associations, and other sources; 


(k) Other matters as appropriate. 


5. to apprise the minister of the advantages and the risks (including, but 
not limited to, direct and indirect costs) associated with the 
recommended model in comparison to the other models and the 
degree of consensus that exists among stakeholders concerning the 
various CSAC models; 


6. to submit a final report by June 1992. 
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Appendix C: Tables 
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Table B: Estimates of Costs of Increasing General Education from 5 to 30% of Program Hours 


Assumptions: 

(1) One year program currently 700 hrs in duration 

(2) Two year programs currently 1400 hrs in duration 

(3) Three year programs currently 2100 hrs in duration 

(4) Full-time Post Secondary Expenditures currently equal $600M 

(5) Grants and tuition fees would increase by .same % to offset additional gen ed. costs 

(6) Faculty salaries account for 50% of operating costs 

(7) Faculty wage bill would increase by same percentage as percentage increase in total program hours 

(8) Percentage increase in non-faculty expenditures would be one-half the percentage increase in program hours. 
(9) Current methods of program delivery remain in place 


Year Programs Two Year Programs Three Year Programs 
{No Change in |All Add. One-Half of |No Change in |All Add. One-Half of |No Change in |All Add. One-Half of 
{Program Hrs. |Gen. Ed. Hrs. |Add. Gen. Ed. |Program Hrs. |Gen. Ed. Hrs. |Add. Gen. Ed. Program Hrs. |Gen. Ed. Hrs. |Add. Gen. Ed. 
are Add-On _ {is Add-On are Add-On _|is Add-On are Add-On _|is Add-On 


Additional G2n. Ed. Hours 

Total Gen. Ed. Hours 

Decrease in Non-Gen. Ed. Hours 
Additional Program Hours 

Total Program Hours 

% \Increase In Total Program Hours 


Total Increase In Grants Required ($M): To move trom 
Total 5 to 20% 20 to 30% 
No Change in Program Hours $0 $0 $0 
All Additional Gen. Ed. Hours are Add-On $161 $84 $77 
One-Halt Gen. Ed. Hours are Add-On $66 $38 $28 
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Appendix D: 
Generic Skills: The BTEC Model 


The Business and Technician Education Council (BTEC) was established in 
the United Kingdom in 1983 as a registered company with a mandate to 
accredit employment-related programs offered by colleges, polytechnics and 
private trainers. Specifically, BTEC is responsible for developing and assuring 
appropriate standards for credentials, as well as encouraging access and the 
opportunities to build upon credentials and enhance the ability of students to 
transfer between programs and institutions. 


BTEC awards credentials and accredits programs at eight levels: First 
Certificates and Diplomas, National Certificates and Diplomas, and Higher 
National Certificates and Diplomas as well as BTEC Continuing Education 
Certificates and Diplomas. The different programs and credentials vary in 
terms of entrance requirements and the number of units required for course 
completion.! Each program consists of a set number of compulsory and 
elective courses (units). That is, 5 units are required for a First Certificate, 8 
for a First Diploma; 10 units are required for a National Certificate, 16 fora 
National Diploma; and 10 units are required for Higher National Certificate, 
16 for a Higher National Diploma. A BTEC First is a prerequisite for a BTEC 
National; a National is a prerequisite for a Higher National. 


Since the Fall of 1986, common skills (what we have termed generic skills) 

have been an integral part of all BTEC First, National, and Higher National 

credentials. Through a two phase development process, BTEC has achieved 
the following objectives: 


* provided a framework of areas compulsory for each program/ credential 


* set national standards for each area including outcomes for each 
common skill by level of credential; 


* provided guidance on delivery, assessment and grading of common skills 
within that competency-based framework. 


BTEC has identified the following seven headings as common skills which 
will appear on each student's "Notification of Performance" (i.e. transcript): 


| A unit, also known as a module, is defined by BTEC as a short course lasting between six and 
eighteen weeks. A unit is also defined as requiring 60 hours of learning for a part-time 
program and 90 hours of learning for a full-time program. (The difference is intended to 
account for the fact that full-time programs include some ‘placement’ hours.) 
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* managing and developing self 

* working with and relating to others 

* communicating 

* managing tasks and solving problems 
¢ applying numeracy 

¢ applying technology 

° applying design and creativity 


The following chart* lists the common skills and competences. 


Common Skill Competence 


Managing and - Manage own roles and responsibilities 
Developing Self - Manage own time in achieving objectives 
- Undertakes personal and career development 
- Transfer skills gained to new and changing 
situations and contexts 
Working with and . Treat others' values, beliefs and opinions with 
Relating to others respect 
- Relate to and interact effectively with 
individuals and groups 
- Work effectively as a member of a team 
Communicating . Receive and respond to a variety of 
information 
. Present information in a variety of visual 
forms 
| 10. Communicate in writing 
11. Participate in oral and non-verbal 
communication 
Managing Tasks and 12. Use information sources 
Solving Problems 13. Deal with a combination of routine and 
non-routine tasks. 
14. Identify and solve routine and non-routine 
problems 
Applying Numeracy 15. Apply numerical skills and techniques 


Applying Technology 16. Use a range of technological equipment and 
systems 


Applying Design 17. Apply a range of skills and techniques to 
and Creativity develop a variety of ideas in the creation of 
new/modified products, services or situations 
18. Use a range of thought processes 


*Source: "Common Skills: General Guidelines (Interim Document)," Business and 
Technician Education Council, 1991, p.10. 
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BTEC requires that the equivalent of one unit in each year of a program be 
devoted to these common skills. Procedures for developing, assessing and 
reporting common skills have been established. As well, BTEC has produced 
an outcome-based approach to defining and assessing common skills which 
differs by level of credential. 


BTEC encourages an assessment of these skills in the workplace or in realistic 
work-based simulations. Other methods of assessment include integration of 
common skills into projects or assignments in all core units as well as the use 
of a log book which is to be used to provide a formative and summative 
record of each learner's achievement. 


BTEC has developed outcome statements under each heading which are to be 
developed at each level. These statements are expressed first as a general 
range statement (a definition of the breadth and limits of applicability of each 
outcome in respect of all BTEC qualifications); second as a range statement (a 
definition of the breadth and limits of applicability of each outcome in respect 
of all BTEC qualifications at each level, i.e. First, National, and Higher 
National); and, third, as performance criteria (lists of criteria by which the 
performance of each competence will be measured and which must all be 
satisfied individually for competence to be demonstrated). 


By way of example, the following are the statements and competences 
provided under common skill "Applying Numeracy" : 


Applying Numeracy 
Competence 15. Apply numerical skills and techniques 


General Range Statement: This competence is about utilizing and applying a 
range of numerical skills and techniques, e.g.: estimating, calculating, 
approximating, use of calculators, use of models and recording data. 


An understanding and appreciation of the role of such techniques will be 
developed through their application in a variety of vocational contexts and 
situation. The use of technology should be introduced where appropriate. 


First : 
Range Statement: At this level, the learner should be able to use and apply 


given skills and techniques to arrive at realistic and valid conclusions within 
the context of adult working life. 


2 British and Technician Council, "Common Skills: General Guidelines (interim Document)", 
1991 pp: 227-35, 50: 
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Performance Criteria: a. Use and application of numerical techniques 


identified. 
b. Appropriate techniques selected and applied. 
c. Numerical information correctly interpreted. 
d. Valid conclusions drawn. — 


National: 


Range Statement: At this level, the learner should be developing the ability to 
identify the need for and to apply numerical skills and techniques within 
differing situations and tasks. The learner should also demonstrate an ability 
to evaluate the suitability of the techniques used and solutions derived. 


Performance Criteria: a. Need for the application of numerical techniques 
identified; . 
b. Appropriate techniques selected and applied; 


c. Numerical information correctly interpreted and 
valid conclusions drawn; 


d. Inappropriate/inaccurate solutions recognized 
and appropriate remedial action taken; 


e. Suitability of results evaluated. 
Higher National: 


Range Statement: The learner will be able to identify the need for numerical 
skills and techniques within the context of perceived or given problems or 
tasks. A wide spectrum of numerical skills and techniques will need to be 
applied to tasks/situations of varying degrees of complexity. 


Performance Criteria: a. Need for the application of numerical techniques 

identified 

b. Variety of numerical techniques identified and 
evaluated; . 

c. Selected techniques applied, tested and valid 
conclusions drawn. 

d. Inappropriate/inaccurate solutions recognized 
and appropriate remedial action initiated. 

e. Suitability of solution evaluated for accuracy and 
fitness for purpose. 
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Appendix E 


The New Jersey General Intellectual Skills (GIS) Assessment: A Summary 


Definition of General Intellectual Skills 

General intellectual skills, as defined by the State of New Jersey, include the 
ability to find, use and present information and data. They include skills in 
analysis, problem solving, critical thinking, quantitative reasoning, and 
written and oral expression.1 


Purpose of GIS 

The fundamental purpose of the GIS instrument is to improve educational 
practices in New Jersey colleges and universities. The instrument is a 
carefully constructed means of evaluating and reporting the skills of groups 
of students rather than individuals.2_ The members of the Student Learning 
Outcomes Committee (one of four committees of the College Outcomes | 
Evaluation Program) reported that the mode of assessment should not be a 
multiple-choice test, but should consist of a common set of tasks which 
would closely resemble what students are required to do in the classroom and 
at work. (A description of several of the tasks which have been developed 
can be found below.) 


Who Is Tested 

A representative sample of 200-300 second year students who have completed 
between 45 - 70 college credits is selected to take the test at each institution 
including vocational colleges, county (community) colleges, state colleges and 
the state university. 


How the Test Was Developed 

The Student Learning Outcomes Committee first established-the skills to be 
assessed. The following key components were agreed upon and guided the 
development of the GIS: 


1 Report to The New Jersey Board of Higher Education from The Advisory Committee to The 
College Outcomes Evaluation Program, October 23, 1987, p.12. 

2 A total of seven tasks are used for the GIS Assessment. However, since Prep purpose of GIS is to 
assess institutional effectiveness rather than each individual, each student is only required 
to complete one task. The scores of individual students from a single institution are combined 
and a composite score reported. Over time, each institution will be able to develop a profile 
of its students’ skills in the aggregate. 
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(1) it would consist of a series of tasks; 


(2) the focus would be on institutional effectiveness, not the individual 
student, i.e., that it would not be a gateway test; that only a sample of 


Students would be needed, and that student motivation to participate 
could be a problem; 


(3) the results would be used to improve the proficiency of students and to 
provide accountability; 


(4) the ideal situation would be to test first year, second year and graduating 
students in order to have a measure of the value-added from college 
Programs. However, due to funding pressures only sophomores have 
been tested as a measure of how proficient students were in general 
intellectual skills after two years of college. Plans to test a sample of 
incoming students still exist, possibly in 1993. 


Educational Testing Service (ETS) was contracted to provide technical 
assistance and management in the development of the GIS. Test 
development took place over two years (1988-89). Tasks were written, revised 
and rewritten by a group of New Jersey college faculty under the supervision 
of ETS developers. As well, standards were set and detailed scoring protocols 
written and revised by the faculty group and ETS. 


Almost 5,000 students were involved in two pilot testings. More that 100 
New Jersey faculty scored the student responses and provided feedback on the 
test, the scoring process and the standards. These comments were used to 
make additional modifications to both the test and the scoring protocol. 


ETS and two outside consultants reviewed the results and found that the GIS 
Assessment was reliable, valid for institutional assessment and that it met all 
requirements for appropriate test development. 


Scorin 

The Weetsamaht is scored by New Jersey faculty who are trained to use a 
method called core scoring, a technique that enables the content and writing 
skills to be scored separately. A six-point scale is used which rates students for 
a correct answer which is very well explained and elaborated (6) to an 
incorrect response (1). The essays are scored a second time on only the 
writing skills demonstrated. 


Reporting 

Scores are reported in aggregate form for the state as a whole, by sector and by 
institution with institutional results presented only within the context of 
sector results. Each institution also receives a printout of how all questions 


were answered. 
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Initial Results fea 
Following the first administration of the New Jersey GIS Assessment in 


March 1990, the following findings were reported: 


(1) More than one-half of the students tested had demonstrated proficiency 
(i.e. achieved the level of proficiency expected of a student completing two 
years of college work) in gathering and presenting information, 25% 
attained some level of proficiency, and 15% lacked proficiency. 


(2) 44% demonstrated proficiency in analyzing information, 41% exhibited 
_ some proficiency and 15% lacked proficiency. In quantitative analysis, 33% 
demonstrated proficiency, 38% somewhat and 29% lacked proficiency. 


(3) 23% of the students did not demonstrate proficiency in writing. 


(4) The administrative procedure, scoring process, data analysis and feedback 
to institutions was found to work well. | 


(5) The motivation to perform well on the task was satisfactory for a large 
majority of students completing the assessment. There was some 
difficulty in getting students to come in for the testing. [It may be noted 
that the ‘institutional’ focus creates some problems in administering the 
exercises, given that students are sampled from all programs. With 
CSAC’s focus on programs, such problems might be minimized.] 


Costs 

The development costs of the GIS were estimated at one million dollars, of 
which $850,000 was paid to Educational Testing Service for its consulting 
services, in areas such as sampling, piloting, item analysis, etc. A New Jersey 
official suggests that development costs for a similar set of exercises would be 
significantly greater “if you were starting from square one”. No estimate was 
offered as to the incremental costs of adding new items, nor of the purchase | 
or licensing costs for use of New Jersey's instruments. 


The cost of administering the GIS to 6,500 students in 1991 was estimated at 
less than $200,000. Of this, more than half was used to pay ‘markers’ a per 
diem (plus travel and expense allowances) for the four days spent on the 6500 
exams. It was suggested that this cost is less than might otherwise be the case, 
given that New Jersey benefits from markers familiar with their scoring 
protocols from experience with the writing sample of the NJ Basic Skills 
Assessment Program administered to all incoming students. However, 
alternatives to such ‘per diems’ may exist. Additionally, while actual costs for 
administering the test might be reduced through the use of regular classes, 
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there are no economies of scale; i.e. nearly all the costs are simply 
proportional to the number of students tested in New Jersey. 


Finally, linking the GIS to “classroom curriculum and paedogogy” has always 
been an aim of the program. While budgetary problems have made it 
difficult to fund additional work in this area at the central level, there are 
some institutional efforts in the early stages. 


Sample Tasks : 
Seven tasks of the 18 which have been developed to date were used in the 
Spring, 1990 administration of the test. Following is a brief description of 


each task. All necessary information is provided for students to complete the 
task. ; 


Teresia and Conland: Students look for significant trends in various 
tables containing data about two fictitious countries, interpret the data, 
speculate on reasons for the trends in the data, and then compare the two, 
basically in terms of their economies. 


Cezanne: Students study colour reproductions of this painter and 
works of two others and are asked to analyze what they see. They are then 
given information about the development of Cubism and asked to 
compare and contrast the specific paintings in light of that information. 


Lemon Sharks: Students receive a map showing the breeding grounds 
of a large fish with data concerning its feeding, growth, migratory habits, 
territoriality, etc. They are then asked to trace the fish's movement and 
calculate population growth, extrapolating from birth rates and other 
information. They then receive information about sturgeon and are asked 
to state the ways in which the purposes and methods of studying the two 
fish might be similar or different. 


Facts: Students receive a list of facts about a country's demographic 
makeup, the education of its citizens, the consumption patterns of its 
people, and the attitudes and values shared by the people. Students are 
then asked to identify relationships among these facts and draw 
conclusions about the society or culture of the country. Students indicate 
what additional information would be needed to support their hypotheses 
about this country's inhabitants. 


Theories of the Universe: Students are given information about 
critical developments within the field of astronomy over a several 
hundred year period that shook cultures of that time. Students are asked to 
evaluate the competing theories based upon their scientific merit, and then 
to account for peoples reactions to these various theories emanating from 
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the beliefs and values that were current at the time these events were 
taking place. Students then comment on the effect these changes had on 
humanity's sense of its place in the universe. 


Sorting: Students receive general information about some of the 
geological processes that shape the world in which we live, and specific 
information about the layers of sediment formed at the bottom of a 
fictitious lake. They are asked to draw upon the information given to 
account for the pattern of sedimentation on the lake bottom. 


Indo-Europeans: Students are asked to be historical linguists in this 
task. They receive information about language families and their origins, 
and about the importance of a language's core vocabulary in providing 
clues to the everyday lives of the speakers of that language. Students 
examine words from the core vocabulary of the original Indo-Europeans, as 
well as additional clues, and attempt to describe the group's original 
homeland and lifestyle. 
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